Many web sites provide a method for public feedback (“posts”) regarding their content. Examples include the many news organizations that allow readers to post comments on online news articles and features. The ability of individual readers to post public comments is generally viewed as a positive feature, contributing to public discourse, enabling the correction of errors, allowing members of the community to express their opinions, and also contributing to a general feeling of “engagement” by members of the community. It has been suggested that people participate in online posting activity (both reading and writing) because they seek information, personal identity, integration and social interaction, or entertainment (see, for example, Nicholas Diakopoulos and Mor Naaman, 2011). Unfortunately, some of the posts may be considered to have relatively low “quality” by several measures. For example, they may contain profanity or hate speech, lack relevance to the associated article, lack factual accuracy, or lack uniqueness. News organizations (and other entities hosting web sites allowing public feedback and postings) would prefer a high level of discourse as well as a dynamic, energetic online conversation that does not stifle discussion or dissent, yet minimizes the number of low quality posts. Many news organizations (and other entities) also hold a commitment to the First Amendment rights protected by the US Constitution. Attempting to satisfy these diverse objectives, in the context of an online forum that generally allows for a degree of anonymity, is recognized as a challenging problem. A popular site could receive hundreds or even thousands of posts per day, making human mediation and pre-screening a costly proposition. Human pre-screening also runs the risk of injecting the mores and prejudices of the human mediator (or moderator) into the screening process—a recognized concern given the desire to promote free and open debate while ensuring civility. Typical methods to address this problem include, inter alia,                a) Requiring a user to open or register an account, with a valid email address, prior to any posting;        b) Allowing users (readers) to “recommend” or “like” a post, or alternatively to “report abuse”;        c) Providing a method to take down or hide postings that are deemed abusive;        d) Providing a method to block certain users (identified as “abusers”) from making publicly-viewable posts;        e) Providing a method to track, and respond to, users who abuse the “report abuse” feature.        
As one example, the Slashdot site provides a threaded discussion on individual news stories with a user-based moderation system. Users have differing levels of “karma” based in part on their prior activity, and some users at any instant of time have the ability to “moderate” comments (posts) of others, increasing or decreasing their score and adding descriptors such as normal, offtopic, flamebait, troll, redundant, insightful, interesting, informative, funny, overrated, or underrated. Paid staff can also moderate comments. When a comment is initially submitted, it is scored from −1 to +2 depending on the user's registration status and prior history (their “karma”). Over time, as moderators do their work, comments can be rated on a scale of −1 to +5. User (readers) can set a threshold level so that they only see comments at or above the selected threshold.
Many sites allow a user to “report abuse”, and comments that receive an excessive number of reports are automatically deleted from the viewable areas. Generally, sites implementing such systems also route the comments identified as abusive to a human reviewer, allow for the human reviewer (generally a paid staff member) to alter the access privileges for the posting user (perhaps blocking all further comments from that individual from public viewing). This also creates a need to “review the reviewer”, and provide a method to identify users who abuse the “import abuse” feature, and deal with their behavior appropriately.
The existing methods contribute to a degree of discipline and civility, and in some cases (such as the quality filter implemented by Slashdot) allow users to screen comments before reading so as to limit their reading to comments that have already been judged to have high quality. The Slashdot approach also allows certain users to achieve high “karma” which allows a higher degree of recognition for users that have contributed productively to civil and high quality discourse in the past. However, the results are imperfect. Low quality comments continue to be posted, and users that have achieved high karma (on sites that support ranking of users) cannot easily transfer that positive recognition to other sites.
In addition to web sites that promote online discussion and dialog, as generally described above, there are web sites that promote online collaboration such as Taskrabbit and StackOverflow. In these environments, users have an incentive to achieve high recognition or high reputation (which may each be qualitatively related to high “karma”), since high recognition or high reputation confers benefits on the site as well as elsewhere. For example, on Taskrabbit, users with high reputation have greater success in competing for tasks. This confers a direct economic benefit. Users with high reputation on StackOverflow have, in some cases, started reporting their StackOverflow reputation on job resumes. However, aside for self-reporting a user's level of recognition in other fora, the ability to transfer one's reputation or karma from one forum to another is awkward and subject to interpretation. Reputation or karma is a measure of how much a given community trusts a given individual. Assuming reputation or karma can be accurately measured with respect to a given community, how should the same measure of reputation or karma be treated with respect to a different community? Ideally, a method would be developed to allow the value of a user's reputation or karma, in one community, to be transferred and evaluated (i.e., interpreted or “weighted”) with respect to a different community.
Recently, an additional problem has been recognized—so-called “fake news” and the difficulty of sorting fact from fabrication in online media. In order to address this problem, recent initiatives have included contextual warnings (e.g., “be careful”), clear display of the source of a post or story, labeling or flagging of suspect links (including a feature allowing human readers to flag seemingly false stories—a form of human moderation and scoring), outside fact-checking, pairing of stories or posts with others that provide “balance”, and scoring methodologies that try to rely on other presumed-credible sources on the same topic. These are useful ideas, but many depend on human moderation or scoring and can be tedious to implement. A machine-implemented method, that could provide some insight into the provenance and likely veracity and authoritativeness of a story or post, would be desirable. One challenge, however, is to do this in a way that is demonstrably even-handed. The machine-implemented method should favor fact-based discussion without excluding posts and stories, and the like, merely because the viewpoints expressed are unpopular. Another challenge is that popular stories (or at least the apparent facts underlying popular stories) are frequently repeated either with or without citation. This makes it difficult to assess the full provenance and veracity of a story, post, or news article, using current methods.
Based on the above discussion, it would be desirable to have further methods to promote a high quality of fact-based discourse. Ideally, posts and stories without demonstrable factual support would be automatically flagged as being potentially unreliable, while still allowing content creators to conveniently create original content with or without reference to a wide variety of already-available sources.
Furthermore, it would be desirable for users that have achieved a degree of positive recognition, on one site, to be able to productively identify that fact on either sites, representing similar or dissimilar communities, while minimizing the chance for abuse or inappropriate interpretation associated with such cross-site recognition. Furthermore, from the standpoint of at least some web hosts, it would be desirable to “monetize” a higher quality of discourse by attracting advertisers and other online services to discussion threads that are recognized as higher quality compared to others. Ideally, these goals would be achieved without significant infringement of a user's First Amendment rights. It is the objective of the present invention to achieve these and other goals, as discussed below.
Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements.